Goto

Collaborating Authors

 maximum deviation






Enhancing reliability in prediction intervals using point forecasters: Heteroscedastic Quantile Regression and Width-Adaptive Conformal Inference

Sebastián, Carlos, González-Guillén, Carlos E., Juan, Jesús

arXiv.org Machine Learning

Building prediction intervals for time series forecasting problems presents a complex challenge, particularly when relying solely on point predictors, a common scenario for practitioners in the industry. While research has primarily focused on achieving increasingly efficient valid intervals, we argue that, when evaluating a set of intervals, traditional measures alone are insufficient. There are additional crucial characteristics: the intervals must vary in length, with this variation directly linked to the difficulty of the prediction, and the coverage of the interval must remain independent of the difficulty of the prediction for practical utility. We propose the Heteroscedastic Quantile Regression (HQR) model and the Width-Adaptive Conformal Inference (WACI) method, providing theoretical coverage guarantees, to overcome those issues, respectively. The methodologies are evaluated in the context of Electricity Price Forecasting and Wind Power Forecasting, representing complex scenarios in time series forecasting. The results demonstrate that HQR and WACI not only improve or achieve typical measures of validity and efficiency but also successfully fulfil the commonly ignored mentioned characteristics.


Efficient Discovery of Significant Patterns with Few-Shot Resampling

Pellegrina, Leonardo, Vandin, Fabio

arXiv.org Machine Learning

Significant pattern mining is a fundamental task in mining transactional data, requiring to identify patterns significantly associated with the value of a given feature, the target. In several applications, such as biomedicine, basket market analysis, and social networks, the goal is to discover patterns whose association with the target is defined with respect to an underlying population, or process, of which the dataset represents only a collection of observations, or samples. A natural way to capture the association of a pattern with the target is to consider its statistical significance, assessing its deviation from the (null) hypothesis of independence between the pattern and the target. While several algorithms have been proposed to find statistically significant patterns, it remains a computationally demanding task, and for complex patterns such as subgroups, no efficient solution exists. We present FSR, an efficient algorithm to identify statistically significant patterns with rigorous guarantees on the probability of false discoveries. FSR builds on a novel general framework for mining significant patterns that captures some of the most commonly considered patterns, including itemsets, sequential patterns, and subgroups. FSR uses a small number of resampled datasets, obtained by assigning i.i.d. labels to each transaction, to rigorously bound the supremum deviation of a quality statistic measuring the significance of patterns. FSR builds on novel tight bounds on the supremum deviation that require to mine a small number of resampled datasets, while providing a high effectiveness in discovering significant patterns. As a test case, we consider significant subgroup mining, and our evaluation on several real datasets shows that FSR is effective in discovering significant subgroups, while requiring a small number of resampled datasets.


Strengths and Weaknesses of 3D Pose Estimation and Inertial Motion Capture System for Movement Therapy

Mohammed, Shawan, Siebers, Hannah, Preuß, Ted

arXiv.org Artificial Intelligence

3D pose estimation offers the opportunity for fast, non-invasive, and accurate motion analysis. This is of special interest also for clinical use. Currently, motion capture systems are used, as they offer robust and precise data acquisition, which is essential in the case of clinical applications. In this study, we investigate the accuracy of the state-of-the-art 3D position estimation approach MeTrabs, compared to the established inertial sensor system MTw Awinda for specific motion exercises. The study uses and provides an evaluation dataset of parallel recordings from 10 subjects during various movement therapy exercises. The information from the Awinda system and the frames for monocular pose estimation are synchronized. For the comparison, clinically relevant parameters for joint angles of ankle, knee, back, and elbow flexion-extension were estimated and evaluated using mean, median, and maximum deviation between the calculated joint angles for the different exercises, camera positions, and clothing items. The results of the analysis indicate that the mean and median deviations can be kept below 5{\deg} for some of the studied angles. These joints could be considered for medical applications even considering the maximum deviations of 15{\deg}. However, caution should be applied to certain particularly problematic joints. In particular, elbow flexions, which showed high maximum deviations of up to 50{\deg} in our analysis. Furthermore, the type of exercise plays a crucial role in the reliable and safe application of the 3D position estimation method. For example, all joint angles showed a significant deterioration in performance during exercises near the ground.


On the Safety of Interpretable Machine Learning: A Maximum Deviation Approach

Wei, Dennis, Nair, Rahul, Dhurandhar, Amit, Varshney, Kush R., Daly, Elizabeth M., Singh, Moninder

arXiv.org Artificial Intelligence

Interpretable and explainable machine learning has seen a recent surge of interest. We focus on safety as a key motivation behind the surge and make the relationship between interpretability and safety more quantitative. Toward assessing safety, we introduce the concept of maximum deviation via an optimization problem to find the largest deviation of a supervised learning model from a reference model regarded as safe. We then show how interpretability facilitates this safety assessment. For models including decision trees, generalized linear and additive models, the maximum deviation can be computed exactly and efficiently. For tree ensembles, which are not regarded as interpretable, discrete optimization techniques can still provide informative bounds. For a broader class of piecewise Lipschitz functions, we leverage the multi-armed bandit literature to show that interpretability produces tighter (regret) bounds on the maximum deviation. We present case studies, including one on mortgage approval, to illustrate our methods and the insights about models that may be obtained from deviation maximization.


Generic Bounds on the Maximum Deviations in Sequential Prediction: An Information-Theoretic Analysis

Fang, Song, Zhu, Quanyan

arXiv.org Machine Learning

ABSTRACT In this paper, we derive generic bounds on the maximum deviations in prediction errors for sequential prediction vi a an information-theoretic approach. The fundamental bounds a re shown to depend only on the conditional entropy of the data point to be predicted given the previous data points. In the asymptotic case, the bounds are achieved if and only if the prediction error is white and uniformly distributed. Index T erms -- Information-theoretic learning, sequential learning, sequential prediction, bounds on performan ce, sequence prediction 1. INTRODUCTION Nowadays machine learning techniques are becoming more and more prevalent in real-time systems such as real-time si g-nal processing, feedback control, and robotics systems. In such systems, on one hand, decisions on the actions are to be made in a sequential manner (sequential decision making); on the other hand, dynamics of the systems as well as the environment that are determined by physical laws will play an indispensable role and must be taken into consideration (interaction with real world).


SpiderMAV Drone Shoots Webs for Perching and Stabilization

IEEE Spectrum Robotics

Perching is turning out to be a very desirable skill for aerial robots. The ability to land on walls or ceilings, rather than having to go to the ground, gives a drone the advantage of being high up in the air (probably why you're using a drone in the first place) without the disadvantage of having to spend a lot of energy not falling. We've seen lots of different perching techniques, most of them bio-inspired, including many different flavors of claws, spines, grippers, and adhesives. One of the best perchers in the animal kingdom (although it rarely gets credited as such) is the spider. And spiders don't just perch: They build infrastructure.